MPI warmup auto #2886
MPI warmup auto #2886
Conversation
…imental/warmup-auto
…ests for automatically picked metric
so that cmdstan has access to the parameter
Don't use first samples for warmup
Changed auto adaptation to specific ignore init_buffer samples
| @@ -214,120 +215,6 @@ namespace mcmc { | |||
| logger.info(message); | |||
| } | |||
|
|
|||
| /* | |||
There was a problem hiding this comment.
can we keep the old v1? It helps when I go back do specific sampler testing. All I need to do then is to switch the sampler call.
| ~adapt_auto_e_nuts() {} | ||
|
|
||
| sample | ||
| transition(sample& init_sample, callbacks::logger& logger) { |
There was a problem hiding this comment.
I've updated the call to mpi adapter, see
. Its logic is easier to understand. It also makes single chain default back to regular warmup.
|
|
||
| if (update) { | ||
| //std::cout << this->z_.inv_e_metric_ << std::endl; | ||
| this->z_.is_diagonal_ = reinterpret_cast<mpi_auto_adaptation<Model> *>(this->var_adapt)->is_diagonal_; |
There was a problem hiding this comment.
I use CRTP to access things like z_. If this is only for MPI you can put it inside var_adapt's learn_metric virtual function. Just want to explain how it's done in other samplers, not something you have to do at this point, since the code is still evolving.
| template <typename Model> | ||
| class mpi_auto_adaptation : public mpi_metric_adaptation { | ||
| #ifdef STAN_LANG_MPI | ||
| using est_t = stan::math::mpi::mpi_covar_estimator; |
There was a problem hiding this comment.
I've updated the dense_e 's covar calculation and no longer use the naive version of mpi_covar_estimator. It's intended to improve communication performance. It's ok for you to use the old version.
| @@ -38,6 +39,11 @@ namespace util { | |||
| static const bool value = true; | |||
| }; | |||
|
|
|||
| template <class Model, class RNG> | |||
| struct has_cross_chain_warmup<mcmc::adapt_auto_e_nuts<Model, RNG>> { | |||
| @@ -141,9 +147,9 @@ namespace util { | |||
| using stan::math::mpi::Session; | |||
| using stan::math::mpi::Communicator; | |||
|
|
|||
| if (Session::is_in_inter_chain_comm(num_chains)) { | |||
| if (file_name.size() > 0 && num_chains > 1 && Session::is_in_inter_chain_comm(num_chains)) { | |||
| const Communicator& comm = Session::inter_chain_comm(num_chains); | ||
| file_name = "mpi." + std::to_string(comm.rank()) + "." + file_name; | ||
| file_name = file_name + "." + "mpi." + std::to_string(comm.rank()); |
There was a problem hiding this comment.
file_name could be output.csv so you'll be looking at output.csv.mpi.1.
yizhang-yiz
force-pushed
the
mpi_warmup_v2
branch
3 times, most recently
from
a7c61ac to
405266d
Compare
Got auto adaptation working with merged mpi_warmup_v2
|
This can close. @yizhang-yiz probably has a more up to date implementation of this elsewhere. |
Submission Checklist
./runTests.py src/test/unitmake cpplintSummary
This pulls in the auto metric from: #2815
How to Verify
This is part of: stan-dev/cmdstan#821
Copyright and Licensing
Please list the copyright holder for the work you are submitting (this will be you or your assignee, such as a university or company): Columbia University
By submitting this pull request, the copyright holder is agreeing to license the submitted work under the following licenses: